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In most cities today, school choice is 
the norm, not the exception. According to 
our recent survey of eight U.S. cities, 55 percent of 
parents said their child attends a school other than 
their neighborhood school. Students in cities are likely 
to have a choice between a wide array of district- 
run magnet and alternative schools, public charter 
schools, and private schools. Nationwide, 6 percent of 
all public school children attend a charter school and 
13 percent of children in traditional public schools 
attend a school their parents chose rather than the 
one they were assigned to, according to the National 
Center for Education Statistics. 


Yet within most cities, district schools, charter 
networks, and independently operated charter schools 
provide different sets of performance information 
to different regulators and groups and are held to 
different performance standards and measures 
of quality. In Detroit, for instance, public schools 
are overseen by a dozen different charter oversight 
agencies, the Detroit Public School District, and the 
state-run Education Achievement Agency, all with their 
own methods of rating schools and holding them 
accountable. In Detroit, St. Louis, and Memphis the 
situation has become so confusing for families that 
local nonprofits and advocacy groups have begun 
publishing their own rating systems to inform parents 
and, in the case of Detroit, to ensure that low- 
performing schools are closed. 


Cities across the country continue to offer more public 
school options to their students. The number of 
charter schools more than doubled in the past decade. 
And dozens of school districts are now employing a 
“portfolio strategy,” expanding and diversifying school 
options while holding all schools, no matter the type, 
accountable for performance. Families living in the 
Cleveland Metropolitan School District, for example, 
can send their children to a charter school, or a 
district school with a focus on digital art, architectural 
design, or environmental studies, or a neighborhood 
school with a traditional curriculum. 

In the midst of this dramatically changing face 
of public education, the need for meaningful, 
comparable, and understandable measures of school 
performance has never been more urgent. Urban 
parents need information to help them make choices 
among a sometimes dizzying array of options. Schools 
need to be able to understand how their performance 
compares to other schools. And government agencies 
overseeing the city’s entire portfolio of schools need 
ways to make a wide range of decisions, including 
which schools should be expanded or replicated, 
receive targeted support, or be replaced. 


To address problems like these, some cities are 
developing new citywide performance systems that (1) 
provide consistent information to families and schools 
and (2) provide one tool to allow government agencies 
to make informed and fair school oversight decisions. 
These systems are referred to as Common School 
Performance Frameworks (CSPF) in this report; they also 
are sometimes called Unified Accountability Systems. 

As part of the Center on Reinventing Public Education’s 
(CRPE) ongoing work to understand and support 
successful portfolio management and district-charter 
collaboration, this report is meant to be a resource 
for city leaders who are considering implementing or 
refining a CSPF across all schools, both district and 
charter. CRPE studied the experiences of six cities, 
including conducting interviews with district and 
charter leaders in Chicago, Los Angeles, Denver, and 
New Orleans 1 and working directly with leaders in 
Memphis and Sacramento to look more deeply at the 
process. See Table 1 for a description of 11 cities that 
currently, formerly, or will soon use various types of 
school performance reports. The goal was to document 
lessons about how these frameworks are developed 
and implemented to inform other cities. 


1 The Louisiana Charter School Performance Compact was developed as a performance accountability tool for all charter schools authorized 
by the Board of Elementary and Secondary Education. This Charter School Performance Compact applies to over 70 percent of students in 
New Orleans as of this writing. 
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To gather the information presented here, CRPE contracted with researchers and practitioners who observed and supported two cities through 
their efforts to develop CSPFs. This guide includes two important resources: 

Lessons from Chicago: Developing a Common School Performance Framework 

This case study, written by Jessica Sutter of EdPro Consulting, describes Chicago’s efforts in developing its School Quality Rating Policy. It is an 
illustration of one city’s approach to a CSPF, including how it overcame common hurdles such as district-charter tensions and data limitations. 

How to Develop a Common School Performance Framework 

Laura Weeldreyer of UPD Consulting and and David Stewart, founder of Tembo, Inc., prepared this practical how-to guide for cities at or 
just beyond the decision point to develop a CSPF. To prepare this report, Weeldreyer draws on her two decades of work in education, both 
as a high-level district administrator and as a consultant to districts and state education agencies. She also spent 10 months facilitating 
the creation of CSPF in a large city and has compiled insights and advice on how to create a framework with input from well-informed key 
stakeholders using reliable and available data that track progress toward agreed-upon goals. Stewart’s contributions include some of the 
more technical aspects of choosing and defining metrics. 

These resources are valuable for any “high-choice” city looking for ways to inform parent choice, government oversight, or strategies for 
portfolio management of the system of schools. They will deeply inform both the technical and political challenges involved in rethinking school 
accountability and provide specific guidance on how those challenges can be overcome. 

They also provide important ideas and lessons to any city or state that is grappling with how to develop rich accountability measures under the 
Every Student Succeeds Act (ESSA). They offer essential guidance about which metrics cities commonly consider, including how they prioritize 
them and what political and technical considerations one should give to them. 


What Is a Common School Performance Framework? 


A CSPF is a tool for measuring the performance of 
an individual school, using a defined set of metrics 
that is common to schools across different agencies 
or governing bodies. CSPFs are shared between 
district-run schools, charter schools, contracted public 
schools, and charter networks within the same city or 
geographic area. The goal is to enable comparisons 
across and between all public schools in the area, 
regardless of who runs them. 

CSPFs are specific to a region (typically a city), as 
opposed to a state school rating or accountability 
system. They allow a city to set their own priorities 
for how school quality is defined and measured that 
go beyond any state definitions or tests, and while 
state rating systems usually rely solely upon standard 
testing data, CSPFs commonly incorporate multiple 
metrics, including non-academic measures such as 
school climate. 

Cities often provide families with common data to 
inform school choice and to provide transparency (for 
example, information in the Washington, D.C., Equity 
Reports). A CSPF goes further, providing an evaluative 
rating that is easy for parents and community 
members to digest. One important goal, although 
rarely achieved, is to ensure there are common 


consequences where schools in the same ratings 
categories get the same treatment across sectors. For 
example, when any public school, district or charter, 
falls below an agreed upon performance standard, it is 
provided with support or intervention services, or it is 
replaced with a school that has been consistently rated 
as high performing and is ready to expand. 

Despite these common goals, cities that have invested 
in fully developed CSPFs have all taken somewhat 
different approaches, as shown below. 

The purpose of the tool. Cities vary to what extent 
they attach consequences to the CSPF Some use the 
data simply to highlight successful schools, place 
informal pressure on low-performers, and inform 
parents. Most, however, use the framework to help 
officials decide: 

D Which schools to target for additional support, 
intervention, or closure. 

D Whether to allow charter schools to replicate or have 
access to district facilities. 

For cities pursuing a portfolio management strategy, 
CSPFs are a key component to ensuring continuous 
improvement and equity across the city. Denver, for 
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example, uses its CSPF to determine where high- 
quality schools are located, which neighborhoods are 
in need of better options, which schools should be 
replicated, which schools are in need of intervention or 
support, and which schools should be closed. 

The types of measures included in the tool. 

Standardized test scores are almost always included, 
but cities differ in whether they emphasize proficiency 
or growth scores or in how they break scores down 
by different student populations or sub-groups. Most 
cities, however, also use other measures of quality, 
such as Advanced Placement enrollment rates, high 
school exit exams, and measures of college readiness 
such as SAT test performance. Beyond academics, 
cities also have decided to incorporate school climate 
that can be gleaned through measures including 
student attendance rates; teacher turnover; teacher, 
student, and/or parent surveys; or rates of use of 
discipline, including suspensions and expulsions. 

Some cities, such as New Orleans and Los Angeles, 
intentionally use very few measures of performance to 
make it clear when schools will face state intervention 
and to ensure that parents can quickly and easily 
assess how a school is faring. With a large number of 
charter schools overseen by the state, the Louisiana 
Department of Education wanted a common way to 
measure school performance for all charter schools 
authorized by Louisiana’s Board of Elementary and 
Secondary Education. Because of the unique nature 
of the New Orleans school system (a nearly all- 
charter school system), Louisiana made sure that 
the autonomy of charter schools was preserved in 
the process; the state intentionally kept metrics very 
simple and understandable, and it made sure not to 
infringe on schools’ individual priorities. Denver, on the 
other hand, uses roughly 80 different metrics to create 
a comprehensive tool that allows school operators to 
evaluate performance using multiple measures yet can 
be streamlined to provide parents with the information 
they need to choose the best fit for their child. 

The process for gathering stakeholder input and 
managing the political process. Every city studied 
for this report gathered input from school staff 
and others, but the process differed in each city. In 
Nashville, an alignment of frameworks across sectors 
evolved with little fanfare. Charter school authorizers, 
with the support of the National Association of Charter 
School Authorizers (NACSA), developed a strong 
performance framework that they then shared with 
the Metro-Nashville Public Schools. The district saw it 
as an improvement on what they had been using and 
decided to use parts of it themselves. In contrast, 


those who led Chicago’s CSPF efforts pointed to 
extensive public engagement, many focus groups, and 
lots of work to help schools understand and calculate 
their scores in advance of public release as key to 
successful implementation. The case study included 
in this report provides more detail on that process. In 
Baltimore, as the sole charter school authorizes the 
district heavily engaged with the charter sector on the 
development of a performance framework that would 
be used to make charter renewal decisions. The end 
result in 2013 was a framework that was aligned, 
but not identical to the framework the district had 
simultaneously developed to hold traditional schools 
accountable. Yet charter school leaders still publically 
supported the new charter accountability tool, thankful 
to have provided input during its development. 

Sacramento’s effort to elicit broad input was easily 
the most comprehensive example studied, with 
tens of focus groups with schools and community 
members and with district officials making dozens 
of home visits to interview families in the city. The 
framework’s design iterated throughout the process 
based on evolving feedback. But public engagement 
alone is not enough. Without the development of a 
plan for sustained momentum or buy-in from the 
incoming administration, the framework did not survive 
a transition of superintendents and other district 
administrative staff. 

Similarly, in Memphis, even after a strong public 
engagement campaign and months of facilitated 
meetings between Shelby County Schools and the 
state’s Achievement School District (ASD), the effort 
to create a CSPF was unsuccessful. Differences 
emerged over metrics and their weights and how 
schools, particularly the lowest-performing ones, 
would ultimately be represented in the framework. The 
ASD has since adopted their own aligned — although 
not identical — accountability frameworks and, as 
of this writing, continue to work through a broader 
engagement with the district. 

The effort in Memphis illustrates how critical it is for 
those involved to maintain a high level of commitment 
throughout the development process. Luckily, a local 
community organization independently developed the 
Memphis School Guide for parents, a tool to compare 
all district and charter schools in the city, after sensing 
that the CSPF process could be derailed. Though 
not as robust as a full CSPF given limited publically 
available data that it draws upon, the Memphis 
School Guide does fill a gap by providing parents with 
easy access to public school performance and other 
information to support parent choice. 
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The technical decisions made about how to 
ensure that the tool is analytically sound and 
understandable. The process by which city leaders 
develop a performance management tool that will be 
used for high-stakes decisions such as charter renewal 
must be able to withstand criticism and challenge. For 
that reason, many decisions need to be made: 

□ What data sources to include? 

□ How to reconcile different data sources across 
district and charter systems or how to collect new 
kinds of data? 


□ How to weight different factors? 

□ Whether to roll up the different metrics into one 
summative “rating” or to present independent 
ratings for specific domains, such as school climate 
and academic achievement? 

Each city studied had different preferences and 
approaches to these questions. The accompanying 
guide goes into depth about the tradeoffs involved in 
different approaches and examples of approaches 
used. 


Avoiding Landmines 


Any time performance data are compiled, presented, 
and especially when used for high stakes decisions, 
much can go wrong. Charter school leaders can 
resist being pulled into a system that they believe 
will limit their autonomies or that will impose a 
new set of performance standards that they did not 
originally agree on. District schools may fear that new 
consequences will be attached to the framework on 
top of current expectations. A variety of community 
leaders, union representatives, and others all may have 
very strongly held views about the “correct” measures 
of school quality. 

CRPE’s research across all six cities offers some 
critical high-level lessons for avoiding landmines and 
making sure CSPFs can be effective long-term tools for 
school improvement throughout a city. 

1. Carefully tend to politics and engagement to 
build long-term sustainability. A CSPF can be 
rushed in implementation or forced on schools 
through dictates, (i.e., a district charter authorizer 
can unilaterally develop and enforce a CSPF) but 
there may be a significant price to pay in political 
backlash or simply a lack of commitment when 
those who pushed the idea move out of the city. 

Those implementing a CSPF should pay close 
attention to the political context in their city and 
consider which individuals and organizations 
need to be involved in the process to improve the 
metrics and to ensure that there is a long-term 
commitment to its use. Having firm commitment 
from the superintendent and key charter leaders 
is essential and should be cultivated and managed 
appropriately. 


Chicago CSPF leaders did extensive outreach 
throughout the city and across different types of 
schools. They sought general consensus on the 
goals and metrics, especially among school leaders. 
Thanks to this extensive engagement during all 
aspects of the process, implementation of the CSPF 
was smooth. 

No one set of performance measures can address 
the specific needs or interests of every group, 
however. Both Chicago and Louisiana leaders made 
it clear that despite an extensive engagement 
process, it would be impossible to accommodate 
the preferences of every stakeholder. 

As the accompanying Chicago case study notes, it's 
wise to check in early with the people who have the 
ability to kill the effort to create a CSPF, know who 
is steadfastly in your corner, focus especially on 
the political system and its players, and know your 
base. It is impossible to engage too much given how 
much these schools have on the line. 

2. Be realistic about the outcomes and metrics of 
the CSPF. The process of developing a CSPF can 
easily take on a life of its own, resulting in a tool 
that attempts to be everything to everyone or that 
tries to provide data for every possible question. 

For that reason, clarity on the goals of the CSPF is 
critical to success. The risk of extensive engagement 
is that the tool becomes watered down, rendering 
it useless. It takes strong leadership to keep the 
process focused for a strong end result. Some cities 
did this by getting community input on a strawman 
tool, others took community feedback and 
developed a tool that would meet basic community 
concerns. Both resulted in solid tools because the 
purpose was clear. 
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Just as important, the metrics used in the CSPF 
must be realistic. It must be feasible to collect the 
data, both technically and politically. Los Angeles 
and Louisiana leaders found that simple metrics 
are easier to agree on and require little to no policy 
changes. Leaders in both places saw the CSPF as 
an opportunity to make a coherent story out of the 
extensive data that are already collected, rather than 
finding ways to collect new data. At any cost, be 
sure that school staff believe in the core assessment 
data. That will make or break efforts to implement a 
CSPF 

3. Be purposeful and focused. Throughout the 
process, it is important for leaders of the process 
to communicate clearly how the framework will be 
used and what outcomes are expected. This helps 
reduce arguments and uncertainty. Transparency 
and effective communication about how the 
community's values and priorities are being 
conveyed through the CSPF will support use and 
gain broader buy-in. 


4. Be thoughtful and gradual on implementation. 

Give schools time to adjust to the new system and 
develop respect for it. A CSPF is useless if it is not 
used for decision making, but decisions will not 
be politically sustainable if the framework is not 
seen as valid, reliable, and sensible. Some cities 
gave schools their data ahead of time so schools 
could see how they would rank or compare in the 
new system. Others allowed charter schools to opt 
in rather than being forced to take part in the new 
system. Still others offered a year of “safe harbor” 
before the new rankings took effect. Across all of 
these examples, time spent letting schools work 
with and understand the data was time extremely 
well spent. The new frameworks in most of these 
cities were generally viewed as better than the old 
systems. And because no rating is ever perfect, 
most cities solicited early feedback and committed 
to making iterations and improvements over time. 


Conclusion 


Through CRPE’s research, we have found that CSPFs 
can be an essential tool for informed decision 
making, accountability, and parent information, but 
only if developed with diverse and strong buy-in and 
with a broad understanding of and commitment to 
how it will be used. The six cities referenced here 
(and many beyond those included here) shared a 
need to provide parents with reliable and consistent 
measures of school quality across all public schools. 
They also share difficult political tensions and the 
pernicious problem of how to sustain such efforts 
amidst leadership turnover and shifting priorities. So 
this work is not for the faint of heart and it is not a 
quick fix for either parent information gaps or holding 


schools to high standards. In places that failed to fully 
implement the CSPF, the hurdles included changes 
in district leadership, stakeholder opposition, and 
poorly conducted community engagement. In those 
places that have continued to use a CSPF there is an 
openness to iteration and a school board that has hired 
leaders who pledge to prioritize and preserve it. 

As cities and states re-think their school accountability 
measures under ESSA, the observations and 
suggestions represented here provide direction 
to mitigate political opposition and build a tool 
that facilitates choice and is representative of the 
community’s values. 
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Table 1: Cities or Regions That Are Using, Have Attempted, or Are 
Developing a CSPF 




Features and Coverage of 



Name of Framework 

Stage of Development 

Framework 

Purpose 

Contact Information 


CHICAGO 


School Quality Rating 
Policy (SQRP) 

Complete; in use by Office 
of Accountability since the 
2013-2014 school year. 

Developed by Chicago Public 

Schools. Provides summative ratings 
for all CPS-run schools and all but 
one charter school. 

Used for 5 purposes: parent 
information, identification of high- 
achieving schools, as a goal-setting 
framework, targeting assistance, 
guide Board decision making. 

Office of Accountability 
sqrp@cps.edu 

DENVER 

School Performance 
Framework (SPF) 

Complete; in use since 

2008. 

Developed by Denver Public 

Schools. Over 80 metrics used. Rates 
all DPS-run schools and all charter 
schools. 

Used for 5 purposes: diagnostic 
tool, performance transparency, 
district decision making, basis 
for accreditation, compensation 
decisions. 

SPF, in the Academic and 
Innovation Office 

spf@dpskl2.org 

LOS ANGELES 

School Performance 
Framework (SPF) 

Complete; in use since Fall 
2012. 

Developed by Los Angeles Unified 
School District (LAUSD) to apply 
to all LAUSD-run schools, and 
independent and affiliated charter 
schools. 

Developed with the intention of 
making previously gathered data 
useful for schools. 

Used as a tool for school- 
level decision making, parent 
information, differentiated 
oversight of schools, charter 
renewal. 

Division of Intensive 

Support and Intervention 

SchoolPerformanceFramework 

@lausd.net. 

LOUISIANA STATE DEPARTMENT OF EDUCATION 

Charter School 
Performance 

Compact (CSPC) 

Complete; in use since the 
2013-2014 school year. 

The CSPC applies to all charter 
schools authorized by the Louisiana 
Board of Elementary and Secondary 
Education, including most schools in 
the Recovery School District. 

Used as a tool for differentiated 
oversight of high- and low- 
performing schools, parent 
information, growth and renewal 
decisions. 

Louisiana State Dept, of Education 
(877) 453-2721 


MEMPHIS 


Common information 
system: Memphis 
School Guide 

In development; in initial 
attempts, leaders were 
not able to create a fully 
aligned CSPF. Memphis 
is still working towards a 
full CSPF. 

A third-party group 
independently developed 
the Memphis School 

Guide parent information 
system, which is complete 
and in use. 

Project was a collaborative effort 
between Shelby County Public 

Schools and the Achievement 

School District (ASD). Would have 
applied to all public schools in the 
city: traditional, charter, and ASD 
schools. 

Memphis School Guide was 
developed by a parent group and 
encompasses all Memphis public 
schools, both district and charter. 

CSPF was intended to be a 
consistent way to measure school 
quality. 

The Memphis School Guide is a 
parent information system, to 
support school choice. 

Memphis School Guide 

hello@ 

memphisschoolguide.org 

SACRAMENTO 





Guide to Success 

Incomplete; project 
was abandoned after 
leadership transition. 

N/A 

Intended to be used as an 
accountability tool. 

N/A 
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Name of Framework 

Stage of Development 

Features and Coverage of 

Framework 

Purpose 

Contact Information 

BALTIMORE* 

Two systems: 

One for charter 
schools (School 
Effectiveness 

Review), one for 
district schools 

Both frameworks are 
complete, and recently 
adopted. 

Baltimore City Public Schools houses 
the frameworks in the Office of 
Achievement and Accountability. 

The frameworks used for district and 
charter schools are closely aligned. 

School Effectiveness Review is 
an aspect of all Baltimore charter 
schools’ renewal process, and is a 
tool for differentiated oversight for 
all schools. 

Achievement and Accountability 
Office 

(443) 984-2000 

CLEVELAND’ 

School Quality Guide 

Development of common 
school profiles are 
complete, which include 
common academic 
metrics. First reports were 
in 2013. 

The Cleveland Transformation 

Alliance, a public-private 
partnership, developed the School 
Quality Guide. The profiles are 
created for all Cleveland district 
schools and most charter schools. 

The School Quality Guide is 
primarily a tool for parent and 
community information. 

Cleveland Transformation Alliance 
(216) 592-2425 

WASHINGTON, D.C.* 

Common information 
through the D.C. 

Equity Reports 
and the Learn D.C. 
website 

No common school ratings 
across district and charter 
schools, but the Equity 
Reports are complete and 
have been in use since the 
2012-2013 school year. 

Developed collaboratively 
between the Office of the State 
Superintendent of Education, the 

DC Public Charter School Board, and 
the Deputy Mayor for Education. 
Reports are created for all D.C. 
district and charter schools. 

The Equity Reports are used 
primarily for parent and community 
information. 

Office of the State Superintendent 
of Education 

osse@dc.gov 

NASHVILLE* 

Two systems: 

district (Academic 
Performance 
Framework) and 
charter 

Both frameworks are 
complete. The Academic 
Performance Framework 
(APF) was adopted in 

2013. 

The APF was developed by Metro 
Nashville Public Schools (MNPS) and 
is applied to all MNPS-run schools. 
MNPS has an aligned framework 
used to evaluate charter schools. 

The APF is used as a tool for 
differentiated oversight of schools 
and support to schools as well as 
parent and community information. 

Metro Nashville Public Schools 

Familylnfo@mnps.org 

TULSA* 

Tulsa Value-Added 
Reports 

No common framework, 
but the district has worked 
with district-authorized 
charter schools to develop 
a framework for charter 
schools. Value-Added 
Reports are used for 
both district and charter 
schools. 

Value-added school reports, 
developed by Tulsa Public Schools 
(TPS), are created for all district 
schools and district-authorized 
charter schools. There is interest 
in a CSPF, but it is not yet in 
development. 

Value-added reports are used for 
informing district decision making, 
identifying best practices and low- 
performance. 

Tulsa Public Schools 

valueadded@ 

tulsaschools.org 


These cities were not part of CRPE’s in-depth CSPF research. 
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Lessons From Chicago: 
Developing a Common School 
Performance Framework 



In 2013, Chicago Public Schools (CPS) created the 
School Quality Rating Policy (SQRP), a system in which 
all schools — neighborhood schools, magnet schools, 
charter schools, selective enrollment schools, and 
special option schools — receive a rating based on a 
standard set of metrics. Schools were given scores and 
ratings (levels 1 through 5) for the first time at the end 
of the 2013-14 school year. 

The SQRP was designed with several different goals in 
mind. CPS wanted a framework that would recognize 
high-achieving and high-growth schools and help 
identify schools in need of targeted or intensive 
support for improvement. This SQRP also would need 
to work as a framework for schools to set goals and 
for the school board to use in their decision-making 
around school actions, such as turnarounds or 
closures. Finally, the tool would need to communicate 
to parents and the public about academic success for 
individual schools and for the district as a whole. 

The school board adopted the SQRP as the tool for 
assessing performance and rating schools according 
to Illinois School Code. All charter schools authorized 
by CPS have accountability provisions in their contract, 
and the school board has decided to use the SQRP as 
the accountability system for charter schools. Thus, 
the school board is responsible for approving the SQRP 
as the accountability system and the CPS CEO is 
responsible for its implementation. 1 

With an understanding that engagement and political 
support would be critical to its success, according 
to sources from CPS, the process for developing the 


SQRP began with many focus group meetings led by 
the Office of Accountability. The planning team invited 
anyone interested in discussing the framework to 
participate; initial discussions included researchers 
from the University of Chicago Consortium on School 
Research, education funders, school leaders, internal 
school district staff, and community partners. While 
these diverse stakeholders were all part of the 
planning committee, there was some tension over the 
group composition. Charter school leaders had a lot 
at stake in the outcome of the framework, but they 
had equal standing with other interested parties. While 
district officials in the Office of Accountability built the 
final framework, input from this initial planning group 
directly informed many aspects of the SQRP 

While CPS expected to use the SQRP as the 
performance accountability tool with charter schools 
going forward, existing charter schools had the option 
of amending their contract language to embed this 
tool. At the outset, 80 percent of the existing 131 
charter schools signed on to have the SQRP function 
as their performance accountability system, and by 
the beginning of implementation, all but one school 
had adopted the SQRP One CPS staff member noted 
that this high level of voluntary participation and 
active engagement by charter schools “made it so we 
couldn’t get lazy” with the development process. The 
single Chicago charter school that did not adopt the 
new system has had the same accountability system in 
their charter contract for 13 years and does not want 
to change. This has led to, as one official noted, a “100 
percent single accountability system — minus 1.” 


1 The Illinois School Code requires the CEO of CPS to monitor the performance of all schools using a criteria and rating system to identify 
schools struggling with achievement in reading and math, attendance, or graduation rates, as well as schools failing to implement required 
programming or improvement plans. Charter schools are exempt from this provision of the school code. 
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While the development of the SQRP was relatively 
smooth, the Office of Accountability still had to 
negotiate a variety of important decisions with input 
from the stakeholder focus groups. 

Outcomes vs. inputs: Focus groups negotiated 
whether all of the SQRP metrics should reflect 
academic “outcomes,” or whether some should reflect 
measurements of school climate or other “inputs” for 
school quality. The Office of Accountability decided, 
with stakeholder support, to heavily weight outcomes 
and include some inputs for “color and context.” While 
academics were much more heavily weighted, these 
inputs also could influence a school’s final score and 
rating category. 

Proficiency vs. growth: The groups also considered 
how achievement proficiency and growth should be 
treated and whether one should be weighted more 
heavily than another. 

Customized metrics for upper grades: When focus 
groups considered a high school-specific measure 
of “college persistence,” they had pushback from 
high school faculty who felt it would be unfair to hold 
them accountable for results after students left. They 
challenged, “Will you hold K-8 schools accountable 
for high school persistence?” This discussion lead 
to a broader conversation of the validity of college 
enrollment data, sourced from the National Student 
Clearinghouse. As one district official noted, these 
validity questions become key when schools are 
“scratching and clawing for 0.1 of a point” on an 
evaluation system. 

As of the 2015-16 school year, the SQRP has been 
in use for two years, and school district staff say that 
the tool has done exactly what they hoped it would 
do. In the years prior to implementing the SQRR CPS 
had used several different accountability systems. The 
most recent system was a three-tier rating scale that 
placed 50 percent of schools in Level 3, the lowest 
tier. According to CPS staff, this scoring distribution 
undermined the accountability system because it did 
not differentiate enough between challenged schools. 
Additionally, since the prior system was designed for 
traditional public schools but was used for oversight 
of charter schools and alternative schools, CPS had to 
add other specific metrics that were more appropriate 
to the needs of such schools. For decision making 
about charter school renewals and closures, CPS used 
a “comparison school metric.” This led to complex 
explanations of the metric and feelings from charter 
schools that the district was intentionally making the 
process opaque. Alternative schools also required a 
new metric to accommodate for limitations of the 


traditional assessment metric. The Prairie State 
Assessment (a standardized test taken by all Illinois 
high school juniors, administered until 2014) assumed 
that all students were attending the assessment school 
for a full year; yet most alternative schools served 
students for only a portion of a school year. A metric 
designed to address this issue only made things more 
complex, undermining efforts by CPS to promote 
transparency of data and decision making for schools. 

The school district and schools themselves are the 
primary audiences for the SQRR with the community 
and parents as a secondary audience. In the first year 
of implementation, CPS announced SQRP ratings 
in October 2014 for the 2013-14 school year; this 
process is repeated annually. The school district cited 
“a few very smart things” the Office of Accountability 
did to make sure that the first year of implementation 
was successful: 

□ It worked proactively to make the development of 
the SQRP a very public process and to generate buy- 
in from charter schools and education stakeholders. 

□ It made sure that schools were not surprised by 
their scores on the SQRP at the public release. All 
schools received “calculators” to allow them to 
prepare their scores on their own and figure out 
where they would be according to the system. 

Additionally, the Office of Accountability now provides 
schools with a preliminary roster for each metric 
during the summer, with the opportunity to review 
and provide feedback before official scores come out. 
This not only helps schools understand how they are 
being scored, but catches any problems with data 
or evaluation. While this process does slow down the 
release of the ratings, one official said, “It is worth it to 
ensure accuracy.” Said another district official, “Three 
years ago, people wanted my head on a platter; now, 
they’re calling me six months ahead of time to say, ‘I 
think we may have a school on the warning list.”' 

After the first year of implementation, district staff 
report that “to a person, for all schools, the SQRP is a 
better evaluation system” for several reasons. Schools 
believed the assessments and metrics included in the 
SQRP were an improvement over the prior evaluation 
tool. The five-tier system allows for greater specificity 
in categorizing schools and avoids grouping all low- 
performing schools into one tier. However, there are 
still kinks to be worked out as CPS and schools move 
forward with the system. For the schools that adopted 
the system, there are still adjustments. Officials noted 
that some charter schools were confused over scores 
generated by the calculators. One result of this is that 
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many schools are now working to better understand 
the NWEA MAP assessment (Northwest Evaluation 
Assessment Measures of Academic Progress test), 
which is the primary test used for assessment, and 
the growth metrics used in the SQRP School district 
officials say that while they think the schools will need 
time to adjust to all the aspects of the new system, 
they are confident that the system will continue to be 
implemented. 

CPS staff offer the following “pearls of wisdom” for 
other cities interested in pursuing a common school 
performance framework: 

□ Check in early with the people who have the 
ability to kill the effort to create a CSPF. Know 
your base and seek out people who are steadfastly 
in your corner. Focus especially on the political 
system in your city and its players. 


□ Get the assessment metrics right. “Do you have 
an assessment everyone believes in? If not — get that 
first and get people comfortable with it. That’s the 
ballgame.” 

□ Involve lots of people so that they can see the 
evolution of the system and the process. Use 

strawman after strawman to demonstrate the 
system and its possible results. It is impossible to 
engage too much given how much schools have on 
the line with an evaluation system. 

□ Be aware of the big risk of losing the forest for 
the trees. A strong system design can start to get 
watered down when trying to address the concerns 
of individual schools. Find a balance when engaging 
stakeholders. 
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How to Develop a Common 
School Performance 
Framework v 


Laura weeldreyer, UPD Consulting 
David Stewart, Tembo, Inc. 




o 

Select Categories 
and Define Metrics 
Carefully 

Identify categories 
in terms of inputs 
(e.g., school 
climate) and outputs 
(e.g., academic 
performance). Choose 
metrics within each 
category, and plan for 
continual refinement. 



©1 

0 

Conduct a Data 
Inventory 

Find out what data 
already exists and 
how it is collected. 

The CSPF should 
be aspirational, yet 
it will take work to 
understand how to 
get reliable data for 
what needs to be 
included. 


Develop the 
Framework 
Mechanics 


1 


Identify how each 
metric is scored 
and calculated 
using categorical or 
continuous scoring. 
Data presentation 
should align with 
the purpose of the 
CSPF and community 
values. 




o 

Test the Framework 

Conduct a pressure 
test to ensure the 
CSPF is working 
technically and 
that the results are 
meaningful. 




0 

Plan for 

Implementation 

Consider different 
rollout scenarios. 
School leaders 
will need time to 
understand and learn 
to use the data, and 
prepare for the CSPF 
to take effect. 


, P „ 
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A Common School Performance Framework (CSPF) 

is a systematic way of measuring and scoring school 
quality and effectiveness based on values and needs 
specific to a district or city’s school system. The 
framework is built from district goals and expectations 
and is developed for a specific purpose and audience. 
Many CSPFs allow for districtwide or peer school 
comparisons. Districts may use a CSPF to manage 
schools, allocate resources and support, inform 
parents and families about school enrollment choices, 
and/or as the basis for charter renewal decisions. 
Strong examples of common school performance 


frameworks exist around the country, including in 
Denver, Chicago, and Louisiana. 

Who would be interested in a CSPF and why? The key 
to this question lies within the name: “common.” A 
CSPF is a framework that spans multiple governing 
bodies, such as a school district and charter schools 
that are authorized by the same district or are located 
within the same geographic setting. Multiple agencies 
must collaborate to design a CSPF to include schools 
that have previously followed different standards of 
accountability. 
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Build With a Clear Purpose 


Before a city engages in any of the processes related 
to developing a CSPF, it should think carefully about 
the purpose. Any city that seeks to do this work must 
be able to answer questions about the purpose and use 
of a CSPF: 

□ What do we need a CSPF to answer for us that we 
cannot answer now? 

□ What will the CSPF be used to do? By whom? 

- How will the CSPF be used once it is developed — 
for accountability, charter renewal or expansion 
decisions, and/or to help parents make informed 
choices? 

- Should the CSPF differentiate for schools in 
turnaround? 

Who is the intended consumer of the information 
and ratings produced by a CSPF? 

Who must have buy-in for this tool to be effective at 
driving the change we envision? 

How will local values and context be incorporated 
into a CSPF? What will it communicate about our 
district as a process and a reporting mechanism? 

How can the CSPF function to satisfy state or 
federal accountability requirements as established 
through the Every Student Succeeds Act? 

The answers vary greatly across cities, and they will 
shape everything from what metrics are considered to 
who is involved in the process of developing the CSPF, 


how the final format is designed and how the results 
get shared; choosing quality metrics is only one of the 
key elements. 

Once a clear purpose has been defined, developers 
should consider how developing a CSPF can be 
integrated and aligned into processes, timelines, 
policies, and structures that already exist in their city. 

□ Is the CSPF replacing a tool that did not work as 
well or was outdated? 

O Is the CSPF something new that will require a lot of 
implementation work? 

□ Does the city or district have strategic goals that 
can be incorporated into the CSPF, creating a 
cohesive and aligned plan for schools? 

Again, the answers will help guide the process that is 
used to develop the CSPF. 

If, for example, a city wants the CSPF to guide how 
schools are managed and supported, or to allocate 
financial resources, it will need to align the CSPF with 
the budget timeline. If the developers want to use the 
CSPF to make charter renewal decisions, they need to 
align the release of the CSPF results with the charter 
renewal timeline and consider what the language in the 
charter contracts says about how renewal decisions 
will be made or what can be used for accountability 
purposes. If the developers want to use a CSPF to 
inform parents, then it must be published in time to 
factor into school choice decisions. Timelines and 
intended use must go hand in hand. 
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Framework vs. School Profile 


Most districts have a lot of information about 
schools that could be very useful (based on the 
intended purpose) but should not be scored as part 
of the CSPF Developers of a CSPF might consider 
creating two kinds of information in parallel: the 
school performance framework and a school profile. 

A CSPF is a group of scored metrics that result in 
some combination of domain and overall ratings. A 
school profile is a collection of critical but unscored 
information and/or metrics about a school that can 
be widely shared alongside the rating of a school’s 
performance and quality. 

The CSPF serves an accountability function: 

□ Defines goals and expectations. 

□ Allows “apples-to-apples” school comparisons. 

□ Helps district administrators offer support and/ 
or differentiates the management structures for 
schools. 

The school profile serves an informational function: 

□ Provides critical information in one spot. 

□ Offers a snapshot of the school. 

□ Is flexible over time as the school grows or offers 
new programs. 

Undoubtedly, some important information about 
schools will not get included in the CSPF If school 
profiles are simultaneously being built, CSPF 
developers can consider including that information in 
the profile. 



Finding Common 
Ground in Memphis 


Shelby County Schools (Memphis, TN) wanted to greatly expand its 
school choice options and provide parents with more information 
on the performance of all the schools available to them as they 
navigate the enrollment process. As part of this multi-year effort, 
Shelby County Schools wanted a tool that could show school 
quality for all of the schools in its city: traditional public schools, 
charter schools, and the schools that the state department named 
as low performing and are now part of the state’s Achievement 
School District. While the Tennessee Department of Education 
and the Achievement School District already had their own 
performance frameworks, no system covered all Memphis public 
schools in a way that conveyed the values of the community. To 
create a tool that all three groups could agree upon, Shelby County 
Schools convened a steering committee with representatives from 
all three groups, as well as community education leaders who were 
committed to transparency and quality for all children in Memphis, 
regardless of what kind of school a child attends or who manages 
it. The CSPF Steering Committee met together for nine months 
to determine the metrics and the scoring system and to agree 
upon a conceptual approach to a CSPF This was a necessary first 
step before the partners felt comfortable engaging in their own 
stakeholder engagement processes. Negotiation and collaboration 
trumped any one party being “right,” as the purpose, and indeed 
the power, of the proposed CSPF was that it would stretch across 
governance structures to bring common ground to the entire city. 
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METRIC: A point of data within a general category to be measured. Defined by “business rules” about what exactly is 
being measured and why, population size to draw from, and what data are included. For example, under the category of 
“Academic Achievement,” a metric might be “PSAT scores,” which could be specifically measured as Average Overall PSAT 
Scores, Percentage of Students Above X in Math, or Percentage of Students with Above X in All Subjects. 

Inputs: Non-academic metrics, such as school climate. 

Outputs: Measurements of test scores, academic achievement, or achievement gap closure; what happens as a result of 
“inputs.” 

WEIGHTS: A way to prioritize different metrics. Particular categories of metrics could count or be “weighted” more heavily than others, 
depending on the priorities, values, and purpose of the CSPF. 

FLOOR and TARGET: Floor is the minimum level for a metric, the score on a particular metric below which a school will get no points. Target is 
the exemplar or standard, the score for a metric that will receive the maximum points possible for that category. 

CONTINUOUS or CATEGORICAL: Flow values/points are scored to the data of each metric. If the value could be any number within a range 
of data, such as average PSAT score, it is continuous. If data are limited to certain categories, such as yes/no, or points are assigned based on 
defined ranges, then data are categorical. 



Terms to 
Know 



Step 1: What Will Be Measured? 


The first step in the process of developing a CSPF is 
defining the categories to include. Most cities include 
categories for academic performance and academic 
growth, often referred to as outputs. Other commonly 
used domains include school climate, college and 
career readiness, equity/achievement gap closure, and 
student perception, referred to as inputs. 

Once categories are defined, metrics are chosen within 
each category. Metrics must be specifically defined, 
including explicit business rules about what exactly is 
being measured, a rationale for including a specific 
metric, N size, what data will populate the metric, and 
when those data are available. For example, a high 
school domain of College and Career Readiness could 
include a metric of PSAT scores. Flowever, PSAT scores 
could mean Average Overall PSAT Scores, Percentage of 
Students Above X In Math, Percentage of Students with Above 
X in All Subjects, or something else entirely. Figure 1 lists 
common categories and metrics. 

CSPF developers can open the lens and start with 
the largest set of possibilities. They should think of 
everything they want to know about school quality and 
how they might know it. But they should also revisit 
the list and make sure each metric connects with the 
purpose — the “why.” The number of metrics included 
and the weights assigned to each metric should be 
based, at least in part, on the stated goals of the CSPF 


If the primary purpose of the CSPF is to make district 
school closure and charter renewal decisions, the 
developers may want to include more metrics than they 
would if the primary purpose is to help parents select 
the right school for their child. 

Next, developers should comb through the metrics 
again and consider whether including certain metrics 
may result in unintended consequences (e.g., number 
of suspensions, which may suppress the reporting 
of suspensions) or whether the way business rules 
are defined for a metric may result in unintended 
consequences (e.g., Advanced Placement (AP) pass rates, 
which may discourage schools from having students 
sit for the AP exam unless they think the students will 
pass, vs. AP pass rates of all 12th graders, which actually 
encourages AP participation). Make sure the metrics 
and business rules are incentivizing the right kinds of 
behaviors in schools and are communicating the values 
that stakeholders feel represent their community. 

A solid list of metrics will demand continued 
refinement with stakeholder input and the school 
district’s value for specific metrics in decision 
making in, for example, how seriously a metric will 
be considered for the kinds of decisions the CSPF 
is intended to inform, as well as any political or 
community ramifications. 
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There is a common desire to have a single CSPF 
answer to every accountability need for city schools: 
how school quality is measured, how parents are 
informed, how supports are allocated, and how 
charters are renewed. The tool will be more effective 
the more focused and specific the purpose and 


planning is. If a city hopes to have a tool serve multiple 
purposes, then it should consider developing different 
versions, one for each purpose. Cities use different 
elements or views of the CSPF for different goals, but 
no one version can equally and fairly achieve multiple 
purposes for multiple audiences. 


Figure 1: Common Categories and Metrics Considered in Common School 
Performance Frameworks 


Category 

Metrics 

Student 

Achievement 

• State or local education administration test scores, different grades, different subjects 

• Proficiency levels at specific grade levels 

• Course passing rates for specific “gateway” courses 

• Tennessee School Success Rate 

• Compared to peer school/district/state 

Student Growth 

• Growth on state test scores over time 

• Use of Student Growth Percentiles (SGP) or Value Added Models (VAM) 

• Achievement level gains or scale score gains 

• Compared to peer school/district/state 

Career and 

College 

Readiness 

• PSAT scores 

• ACT or SAT scores 

• Participation rates for PSAT, ACT, and/or SAT 

• Percent of 9th grade cohort completing high school within four years 

• Percent who meet state university entry requirements 

• Achievement on state high school exams 

• Enrollment and passing rates for AP or International Baccalaureate coursework 

• Early childhood literacy rates 

• Compared to peer school/district/state 

• Percent FASFA completion 

School Climate/ 
Culture 

• Survey ratings from parents, teachers, and/or students 

• Attendance 

• Chronic absenteeism rates 

• Staff attendance 

• Expulsion/suspension rates 

• Student persistence/re-enrollment averages 

• Compared to peer school/district/state 

• Participation in leadership activities, extracurricular activities, enrichment activities (#/% of students) 

Achievement 

Gap Closure 

• Gap closure for various sub-groups (English as Second Language students, free and reduced-price meal students, minority 
students) 

• Compared to peer school/district/state 

Transformation 
or Gateway* 

• Catch up and keep up growth 

• Continuously enrolled growth (measuring students who have been enrolled 2+ years) 


* District-specific data that have been showcased as key to/highly correlated with ultimate achievement 
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Clarity 
Around 
Goals in 
Louisiana 



The Louisiana State Department of Education (LSDOE) wanted a common school performance 
framework that would help it balance the need to provide accountability and differentiated support to 
the state’s charter schools while maintaining the autonomy of its diverse system of schools. To facilitate 
the development of its Charter School Performance Compact (CSPC) and to minimize controversy in 
the process, LSDOE took the perspective that simplicity and clarity were key. LSDOE leaders defined a 
clear purpose of what the CSPC was and was not: it was about accountability, oversight of the system, 
and parent information. Larger conversations about fundamental charter school issues or the specific 
agendas of individual schools and advocacy groups were certainly important, but they should happen at 
another time. 


As a result of setting this intention, LSDOE, along with 16 charter and consulting firms, decided to keep the categories of metrics 
very limited, focusing on academic, organizational, and financial metrics as a baseline to compare schools. They chose to operate 
within the existing policy framework for school accountability and develop a tool that was simple for schools to use and understand, 
with a small number of indicators and clear consequences and outcomes for schools. 


■Mj Step 2: Conducting a Data Inventory 


Once CSPF developers have a list of categories and 
metrics, they can conduct an inventory of what data 
already exist, how they are collected, who or what 
office collects them, and when. The answers may vary 
for districts and charter schools. Depending on local 
policies and specific state charter laws, the developers 
may be dealing with multiple data systems. If so, they 
will need data-sharing agreements so that one agency 
can collect data from all involved parties for the CSPF 
The technical mechanics of data sharing and transfer 
will need to be worked out. 

Timelines are important because they will relate back 
to the purpose. If, for example, the purpose dictates 
that CSPF results need to be available by May, and the 
district wants to use a CSPF to evaluate principals or 
make staffing decisions, then it will have to think about 
when data are available for those kinds of decisions 
and what can be included in a CSPF Timelines can also 
effect availability of metrics: based on timing, certain 
metrics may not be able to be included, or some data 
may have to lag by a year to make sure the purpose of 
the CSPF is met. 


This is the second step because CSPF developers 
should not be bound to what data are currently 
available; the CSPF should be aspirational. In other 
words, if a city does not have the data now, how can it 
get them? This might mean introducing an agreement 
to administer a specific survey or even agreements to 
collect different kinds of data that have not been used 
before, such as the completion of the Free Application 
for Federal Student Aid (FAFSA), student engagement 
data, or ratings from school reviews. These are 
examples of data that might currently be collected by 
other offices or programs and not traditionally used 
as part of an accountability framework. There might 
be metrics that require kinds of data that the district 
collects but the charter schools do not, or vice versa. 

In Memphis, the schools in the Achievement School 
District, for example, administered NWEA’s MAP 
assessment, but Shelby County Schools did not at 
the time . 1 Developers may also decide there is a need 
for data that do not currently exist but are valued or 
prioritized in the development of the CSPF. The district 
and charter schools will then have to collaborate on a 
plan for how to collect such data. 


1 In school year 2015-2016, Shelby County Schools began administering MAP testing. 
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Step 3: The Mechanics 


The mechanics of the metrics must be carefully 
thought through. Once metrics have been identified 
with the inventory of current data in hand, developers 
will have to consider how the metrics are actually 
scored. This means formally documenting the business 
rules and definitions for each metric (see Figure 2 for 
examples). This kind of documentation will be useful 
down the line in ensuring that the CSPF is transparent 
and for training and communications purposes. 

Along with the business rules, developers must agree 
upon the scoring range for each metric and how the 
scores will be calculated. Districts have approached 
this question differently. There are many examples of 

categorical or continuous scoring. 

Another approach to scoring that is gaining increased 
appeal nationally is to set floors and targets for each 
metric — for example, define the floor (the score on 


a particular metric below which a school will get no 
points) and the goal line of a particular metric, or the 
target for a school. For instance, if the CSPF includes 
a Student Attendance Rate metric and a 4-Year Cohort 
Graduation Rate metric, and each metric is worth 10 
points, should a school that achieved 80 percent on 
each metric receive 8 out of 10 points? Probably not. 

A school with an attendance rate of 80 percent may 
not be awarded any points, but a graduation rate of 
80 percent may be worth 8 or 9 of the 10 possible 
points. To optimize the ability to award points based 
on meaningful scores and to equate different types of 
metrics, developers may want to consider applying a 
floor and a target to each metric. 

Using attendance as an example, consider the possible 
ways to score attendance in a CSPF, as shown in the 
inset. 


Figure 2: Examples of Metrics and Business Rules 


Category 

Indicator 

Indicator 

Number 

Measure 

Grade 

Level(s) 

What does this mean? 

Academic 

Performance 

Achievement 

1.A.1 

Composite proficiency rate 
for reading/language arts 

3-8 

The percentage of eligible students who earned scores of 
Proficient or Advanced on the TCAP in Reading/Language Arts 

Achievement 

1.A.2 

Composite proficiency rate 
for math 

3-8 

The percentage of eligible students who earned scores of 
Proficient or Advanced on the TCAP in Math 

School 

Climate 

Students 

3.A.2 

Overall attendance rate 

K-12 

The average number of days students attend school divided by 
the average number of days students are enrolled 

Students 

3.A.3 

Overall student persistence 
rate 

K-12 

The percentage of eligible students who are enrolled at a school 
on the audited enrollment date and June 1 in the same school 
year 

Students 

3.A.3.a 

Student persistence 
rate for economically 
disadvantaged students 

K-12 

The percentage of eligible students who are enrolled at a school 
on the audited enrollment date and June 1 in the same school 
year 
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Attendance has a score range of 0 to 100 percent. Generally, the state sets a standard for 
satisfactory attendance, depending on grade levels served. In Maryland, for example, the 
state standard for elementary schools is 96 percent. 


STUDENT 
ATTENDANCE: 
Two Different 
Approaches 



Categorical Approach 

To define categories, a city might look at the range of attendance data and see that no 
school has an annual attendance score of less than 75 percent. So they might define the four categories of attendance scores as 
follows: 0 points = below 75 percent, 1 point = 75-85 percent, 2 points = 86-90 percent, 3 points = 91-95 percent, and 4 points = 96 
percent and above. 


Floors and Targets Approach 

To define the floor, a city might look at the range of attendance scores for all of the schools and see that no school in the district 
has an annual attendance score of less than 75 percent. So 75 percent will be the floor-for example, if a school’s attendance rate is 
below 75 percent, the school will receive 0 points within the CSPF. The target could be defined as the state standard of 96 percent: a 
school that meets the state standard will receive the maximum points possible. A school that has a 97 or 98 percent attendance rate 
will not get extra points. In this example, then, the point value for attendance will be assigned within a range of 75 to 96 percent. 


Many different methods can be used to determine 
floors and targets, and there is no right or wrong 
answer. Some school systems use the actual 
distribution of scores for all schools over a one- 
to three-year period and assign the 10th or 90th 
percentile as the floor and target respectively. Other 
models have used historical data to calculate the 
mean and standard deviation for each metric and 
assigned floors and targets that represent two standard 
deviations below and above the mean. Another fairly 
common option is to assign floors using one of the 
models just described and to assign targets that are 
aligned to district or authorizer priorities. For example, 
the 90th percentile for math proficiency rates for the 
past three years might be 84 percent, but the district 
or authorizer may choose to set the target at 100 
percent to align with a stated goal. 

To put the metrics together in such a way that the 
CSPF tells a story about the school’s quality and 
effectiveness, developers must make numerous 
decisions about how the CSPF works as a whole: 

□ Scaling: How is the SPF scored (A-F type grades, 
stars, numbers, or labels)? 

□ Rollup: How are the metrics rolled up into a score or 
scaled individually/by category? 

- Will domain-specific scores be reported in 
addition to the overall grade (e.g., sub-grades 
for school climate, academic growth, and 
achievement gap closure, etc.)? 


□ How is the CSPF presented and displayed: school 
report cards, summary scores, PDFs, district or 
school website, interactive comparison tool? 

The best way to answer these questions is to 
continuously return to the purpose of the CSPF and 
check answers against that purpose, thus ensuring 
alignment. If the CSPF is intended to inform parents, 
an online tool linked to school enrollment processes 
and forms might make the most sense. If school 
accountability and the allocations of resources 
(management, support, or financial) is the goal, then 
an annual release of school report cards or school 
progress reports could be the answer. 

The final mechanical decisions have to do with weights 
for each category and within each category, for each 
metric. This is another opportunity to communicate 
values. What a city values most will be weighted 
the heaviest. Developers can start by assigning 
weights to categories. Once those look like the right 
communication of your values, developers can move 
on to weighting the individual metrics. Developers will 
have the chance to test these weights in the next phase 
of work. Most districts choose to weight academics 
most highly, but even that has nuance: Will they weight 
academic performance more highly than academic 
growth, or the opposite? This is also an opportunity to 
revisit whether the CSPF creates the correct incentives. 

For example, in lower-performing school systems, 
a tendency often exists to weight growth or student 
progress more than academic achievement. Within a 


Apples to Apples: Common School Performance Frameworks as a Tool for Choice and Accountability 


18 


Most states have an accountability framework for measuring a school’s 
performance. Cities developing a CSPF should ask themselves what they are 
trying to do differently. This is a chance to be thoughtful about how a locally 
developed framework can complement what is provided by the state. As a city 
enters into the process of developing and then building a CSPF, it should keep 
these guiding questions in mind: 


Thoughtful 

Accountability 






Equitable 

• Are the measures reliable (e.g., 
standardized test scores, growth 
measures)? 

• Are the measures systematically 
biasing certain schools (e.g., 
performance vs. growth)? 

• Are the measures creating the right 
incentives (e.g, suspensions, AP pass 
rates)? 


Actionable 

• Are there clear rewards and 
interventions? 

• Are parents able to use the tool to 
make informed enrollment decisions? 

• Are there different expected actions 
associated with the different overall 
scores? 

• Can the CSPF be used to focus school 
supports/interventions? 


Transparent 

• Do key people understand how the 
framework works and are they able 
to explain it easily and clearly? 

• What are the tradeoffs between 
technical precision and access/ 
understanding? 


particular domain, developers may want to count some 
metrics (e.g., Proficiency Rate in Mathematics) more than 
other more aspirational metrics (e.g., Percentage of 
Students Earning Scores of Advanced in Mathematics). 

Once you have a list of metrics, weights, floors, and 
targets, all that is left with regard to scoring is figuring 
out how to roll everything up into one or more scores 
or ratings. Developers may choose to use words 
(e.g., Model, Not Meeting Standards, Priority, Focus, 
Exemplary), letter grades, or tiers (e.g., Tier 1, Tier 2, 
Tier 3), but regardless of how the different outcomes 


are labeled, developers will need to figure out what cut 
scores to assign to each possible outcome. One option 
is to simply divide the total number of possible points 
by the total number of different outcomes so that each 
overall level is based on the same size scoring range. 
Another option is to simulate the results using different 
cut scores to end up with a distribution that makes 
sense. Additionally, developers will need to decide 
whether to report domain-specific grades or ratings 
in addition to the overall score, or whether they will 
simply use one summative index. 
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Step 4: Conducting a Pressure Test or Trial Run 


Now that the mechanics have been worked through 
and all the pieces fit, it is time to test the whole CSPF 
as a working framework. Developers must conduct a 
pressure test of the CSPF to ensure that it is working 
technically — for example, the data are being pulled in 
from the correct sources and are being aggregated and 
scored according to established business rules, and 
that weights are being applied correctly. A pressure 
test also can be used to look at the distribution of 
school scores: do scores match intuition and pass 
a “sanity test?” Developers can consider what it will 
look like if these results get published in the local 
newspaper. Will they make sense to the average citizen? 
Specifically, do the scores contradict ratings from the 
state accountability system or any other accountability 
measures? If so, developers will need to think carefully 
before releasing the scores. Developers of the CSPF 
may be comfortable with the differences (especially 


if they are the result of weighting growth more highly 
than performance, for example) but must be prepared 
to answer questions from the school community. 

The pressure test can also be used to verify scaling 
distinctions. For example, does the cut score for each 
step in the scale (top, average, failing schools) make 
sense, or does it create meaningless and inactionable 
distinctions? Are the results actionable, do they 
support sound decision making and potentially provide 
good incentives? Are the metrics strongly correlated? 
Are there redundancies or skewing? These questions 
can only be answered by running a simulation of 
the CSPF scores. In addition, the pressure test can 
produce results that can be used to brief a school 
board or to train and communicate with schools prior 
to scores becoming public. 


Step 5: Gearing Up for Implementation 


While it is easy to focus on the technical aspects of 
building a CSPF the best efforts focus equally on the 
process. From the start, there should be involvement 
from various stakeholders, including those who will 
make use of the data produced by the CSPF and 
those who will be affected by the results (such as 
school leaders, school management organizations, 
or parents). Stakeholder input ensures both that the 
framework, metrics, and weightings chosen reflect the 
needs, values, and reality in that city and that there 
is broad understanding of the purpose of the CSPF 
and how it will be used. A pressure test has been 
conducted, and the engagement and communications 
efforts must expand greatly. CSPF developers must 
create engagement and communication plans to get 
wide buy-in from key stakeholders in order to roll out 
the CSPF. These plans can also include training on the 
categories, metrics, and scoring of the CSPF 

A city may want to consider whether to roll out 
the CSPF for a period of time before applying 
the consequences or interventions for schools. 
Stakeholders will need time to understand the data, 
question the data and, ultimately, to learn to use the 


data. Some districts introduce it with a no-stakes 
rollout for one academic year as a soft launch. It 
may increase buy-in if stakeholders can learn about 
the CSPF without immediately worrying about 
consequences. Another way to assist school leaders in 
understanding the framework and preparing for rollout 
is to provide a score “calculator,” or a score projection 
tool, ahead of releasing the final scores. This allows 
schools to predict their scores, work proactively to 
improve their scores, and correct any mistakes in data 
or information before scores go public. 

The school district and charter organizations also will 
need time to learn about the CSPF internally and to 
integrate it into existing processes for performance 
management, school support, and/or charter renewals. 
The CSPF communication and engagement strategy 
must include the central office as well. This will be a 
new way of thinking about school quality for all parties. 
To ensure consistent and positive communication 
about the CSPF, developers should make sure that 
the central office staff has a full and common 
understanding of the framework. 
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Conclusion 


As illustrated in the case study on Chicago’s School 
Quality Rating Policy, as well as in examples embedded 
in this guide, no process of developing a CSPF is 
exactly alike; the development process is just as 
important as the CSPF itself in ensuring smart use. A 
CSPF can be a tool for facilitating informed decision 
making for both parents and school system leaders, 
as well as for targeting supports, interventions, and 
rewards for schools. But it is only successful when 
built with a shared understanding among stakeholders 


of how schools will be measured, the purpose of 
the tool, and how it will be embedded into decision 
making processes. Through thoughtful consideration 
during the development process of who needs to be 
engaged and how, as well as the proper metrics to 
align with its purpose, cities and states embarking on 
the development of a CSPF can create a tool that is 
understandable, supported by school leaders, conveys 
the values of the city, and supports a truly citywide 
system of schools. 



Find More 
Information 


Cities and states can use this guide to develop an effective and fair tool for 
measuring school quality across district and charter lines. The following 
organizations and resources also provide relevant and useful information 
and services: 


• National Association of Charter School Authorizes’ (NACSA) Core Performance Framework is an in-depth guide to 
creating a performance framework, developed based on NACSA’s research on best practices for charter school authorizers 
as well as experiences with supporting ten authorizers through a process of development of performance frameworks. 
NACSA also has a library of resources on performance frameworks, including case studies on cities’ experiences and 
presentations on accountability, which can be found here. 


• Columbia University’s Center on Research and Policy Leadership (CRPL) provides consulting support for organizations 
interested in developing CSPFs to be used by state departments of education, cities, school districts, charter organizations, 
advocacy groups, and other nonprofits. CRPL can be reached at cprl@law.columbia.edu. 


• Tembo, Inc. provides consulting for data management, analytics, and telling a story through data for school systems. 

Their work has included projects such as school performance frameworks, equity reports, school snapshots, and more. 
Tembo, Inc. was founded by David Stewart, who provided technical support to two school districts for this project and is a 
co-author of this how-to guide. 


• UPD Consulting provides consulting on performance management for public sector organizations, including school 
districts, advocacy groups, or charter organizations. Laura Weeldreyer, a consultant at UPD, acted as a facilitator for the 
development of one city’s CSPF for this project and co-authored this how-to-guide. 
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